Solving Risk-Sensitive POMDPs With and Without Cost Observations
نویسندگان
چکیده
Partially Observable Markov Decision Processes (POMDPs) are often used to model planning problems under uncertainty. The goal in Risk-Sensitive POMDPs (RS-POMDPs) is to find a policy that maximizes the probability that the cumulative cost is within some user-defined cost threshold. In this paper, unlike existing POMDP literature, we distinguish between the two cases of whether costs can or cannot be observed and show the empirical impact of cost observations. We also introduce a new search-based algorithm to solve RS-POMDPs and show that it is faster and more scalable than existing approaches in two synthetic domains and a taxi domain generated with real-world data.
منابع مشابه
Probabilistic Planning with Risk-Sensitive Criterion
Probabilistic planning models and, in particular, Markov Decision Processes (MDPs), Partially Observable Markov Decision Processes (POMDPs) and Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) have been extensively used by AI and Decision Theoretic communities for planning under uncertainty. Typically, the solvers for probabilistic planning models find policies that min...
متن کاملFiltered Fictitious Play for Perturbed Observation Potential Games and Decentralised POMDPs
Potential games and decentralised partially observable MDPs (Dec–POMDPs) are two commonly used models of multi–agent interaction, for static optimisation and sequential decision– making settings, respectively. In this paper we introduce filtered fictitious play for solving repeated potential games in which each player’s observations of others’ actions are perturbed by random noise, and use this...
متن کاملDelayed observation planning in partially observable domains
Traditional models for planning under uncertainty such as Markov Decision Processes (MDPs) or Partially Observable MDPs (POMDPs) assume that the observations about the results of agent actions are instantly available to the agent. In so doing, they are no longer applicable to domains where observations are received with delays caused by temporary unavailability of information (e.g. delayed resp...
متن کاملEfficient Approximate Value Iteration for Continuous Gaussian POMDPs
We introduce a highly efficient method for solving continuous partially-observable Markov decision processes (POMDPs) in which beliefs can be modeled using Gaussian distributions over the state space. Our method enables fast solutions to sequential decision making under uncertainty for a variety of problems involving noisy or incomplete observations and stochastic actions. We present an efficie...
متن کاملExecution Monitoring to Improve Plans with Information Gathering
There has been much recent interest in planning problems with deterministic actions but stochastic observations. Examples include Mars rover planning, robot monitoring tasks and the Rocksample domain from the planning competition. However, theoretical results show that in general these problems are as hard as solving partially observable Markov decision problems (POMDPs). We propose an approach...
متن کامل